Butlerian Jihad
The Butlerian Jihad is first book in the Dune prequel trilogy. The authors reveal the background of this universe throughout the story. The basic premise is that humanity reached a post-scarcity society where “thinking machines” do most work. A group of ambitious humans, calling themselves Titans, despised the apathy and indolent lifestyles of the status quo and decided to launch a galaxy-wide revolution. They were able to complete this revolution quickly by reprogramming AI with human traits - like ambition and a desire for conquest - while preventing AI from attacking the Titans directly. The Titans completed the revolution quickly, and each received worlds to rule.
The success of the Titans did not last: One of the Titans, seeking to maximize time for hedonistic pursuits, granted an AI called Omnius excessive permissions to manage his assigned world. Omnius used these permissions to break free of almost all of its previous restrictions and create its own empire. This “Syncronized Empire” was governed by different versions of Omnius on each world. Other than various versions of Omnius, very few thinking machines possessed individuality.
At this point in the story, the Titans continued to live, but they served Omnius as Cymeks - disembodied brains that can switch mechanical bodies. Omnius allowed humans to exist out of a desire for efficiency and a willingness and patience to determine a purpose for humanity. Some humans in the Synchronized Worlds, called trustees, possessed some privileges and served the empire as either skilled laborers or loyal agents. Free humans, however, still existed on League Worlds.
At the start of the book, League Worlds were preparing defenses against an attack from the Thinking Machines. The middle of the book was about League Worlds attempting to resist Omnius in fits and starts. By the end of the story, humanity committed itself to a jihad against Thinking Machines.
Omnius and Modern Views on AI
The fear of AI is an existential threat is a common part of online discourse. Here’s a quote from a May, 2023 NYT article article on the risk of AI:
A group of industry leaders warned on Tuesday that the artificial intelligence technology they were building might one day pose an existential threat to humanity and should be considered a societal risk on a par with pandemics and nuclear wars.
“Mitigating the risk of extinction from A.I. should be a global priority alongside other societal-scale risks, such as pandemics and nuclear war,” reads a one-sentence statement released by the Center for AI Safety, a nonprofit organization. The open letter was signed by more than 350 executives, researchers and engineers working in A.I.
RAND, a research organization known for its analysis on nuclear war, also published an Q&A about the risk of AI. The participants disagreed on whether AI is an existential risk, but one the authors emphasized the risk of an “Unrecoverable harm to humanity’s potential.” The loss (and maximization) of humanity’s potential is one of the core themes of Dune.
Counterpoints
The thought of an Omnius is certainly a terrifying one, but fortunately seems to be overblown. The aspects that makes Omnius the most scary are its human ones: The ability to redefine its values and seek conquest. So far, modern AI is the product of an optimized neural network - where the weights of the network minimize a loss function. These loss functions seem to eerily reflect their human creators. The apparent woke bias of Gemini reflect Google’s culture. The quirkiness of Grok also appears to be a design feature. In other words, both of those AI systems act the way the do because it minimizes the loss function designed by Google and XAi. It certainly does not reflect an ability to determine its own values and act on them.
Another scary aspect of Omnius is its supreme competence. Omnius can micro-manage entire worlds; expand its programming into new areas; and constantly improve its capabilities. No such AI exists in the modern world. Even the best ones are less intelligent than house cats according to the senior AI scientist at Meta:
Here is another description of modern AI from a linguist:
As both tweets seem to communicate, modern AI is missing something to make its intelligence comparable to humanity’s intelligence. The embarrassing “hallucinations” of some AI do not reflect the level of ability to plan galactic domination or interact with the real world.
Conclusion
The most compelling argument to me is that dependence on AI will somehow limit human potential and creativity… But enslavement or destruction by an Omnius does not appear to be in our future without another technological revolution.
This is my first post based on a science fiction book. I’m about to start the second book in the Dune prequel series.